Penetration Testing: A Duet

Penetration testing is the art of finding an open door. It is not a science as science depends on falsifiable hypotheses. The most penetration testing can hope for is to be the science of insecurity not the science of security inasmuch as penetration testing can at most prove insecurity by falsifying the hypothesis that any system, network, or application is secure. To be a science of security would require falsifiable hypotheses that any given system, network, or application was insecure, something that could only be done if the number of potential insecurities were known and enumerated such that the penetration tester could thereby falsify (test) a known-to-be-complete list of vulnerabilities claimed to not be present. Because the list of potential insecurities is unknowable and hence unenumerable, no penetration tester can prove security, just as no doctor can prove that you are without occult disease. Putting it as Picasso did, "Art is a lie that shows the truth" and security by penetration testing is a lie in that on a good day can show the truth. These incompleteness and proof-bydemonstration characteristics of penetration testing ensure that it remains an art so long as high rates of technical advance remains brisk and hence enumeration of vulnerabilities an impossibility. Brisk technical advance equals productivity growth and thereby wealth creation, so it is forbidden to long for a day when penetration testing could achieve the status of science. That penetration testing is an art means that there are artists. In deference to those artists, they range from virtuosos to mules. At the low end, automation (tractors) is replacing brute labor (mules). Automation is the handmaiden of commoditization, and there is little doubt that the penetration field is fully commoditized at the lower levels of art. At that low level, scanning systems steadily expand the scope and coverage of what they automate. That those same scanning tools can be deployed for evil purposes is irrelevant unless you are in the newspaper business. As Sherlock Holmes said to Watson (holding a scalpel), "Is it not surprising that the tools of healing and the tools of crime are so indistinguishable?" No, it is not surprising a good tool is a policyneutral force multiplier and it is intent, that is to say character, that determines the outcome of that force multiplication. Penetration testing is therefore good or bad depending on the intent of its practitioner and of the recipient of its results. We confine this article to penetration testing where the intent is good (the only kind one has to pay for).